Codes on Graphs: 
Observability, Controllability and Local Reducibility 



G. David Forney, Jr. and Heide Gluesing-Luerssen 



o: 

< 

O 
m 






> 

in 



en 
rn 

o 



X 



Abstract — This paper investigates properties of realizations 
of linear or group codes on general graplis tliat lead to local 
reducibility. 

Trimness and properness are dual properties of constraint 
codes. A linear or group realization with a constraint code that is 
not both trim and proper is locally reducible. A linear or group 
realization on a finite cycle-free graph is minimal if and only if 
every local constraint code is trim and proper. 

A realization is called observable if there is a one-to-one 
correspondence between codewords and configurations, and con- 
trollable if it has independent constraints. A linear or group 
realization is observable if and only if its dual is controllable. A 
simple counting test for controllability is given. An unobservable 
or uncontrollable realization is locally reducible. Parity-check 
realizations are controllable if and only if they have independent 
parity checks. In an uncontrollable tail-biting trellis realization, 
the behavior partitions into disconnected subbehaviors, but 
this property does not hold for non-trellis realizations. On a 
general graph, the support of an unobservable configuration is 
a generalized cycle. 

I. Introduction 

The theory of minimal reaUzations of linear codes and sys- 
tems on a conventional discrete time axis has long been well 
understood. In classical minimal realization theory, the con- 
cepts of observability and controllability are central; indeed, 
the classical mantra is "minimal = controllable + observable." 

Moreover, as is well known, classical minimal realization 
theory extends straightforwardly to realizations of linear (or 
abelian group) codes or systems on cycle-free graphs H. 
Given such a graph, the State Space Theorem defines a unique 
minimal "state space" for every internal ("state") variable. 

On the other hand, powerful codes that can approach 
capacity with iterative decoding (e.g., low-density parity-check 
(LDPC) or turbo codes) are necessarily defined on graphs that 
have cycles. Progress in extending minimal realization theory 
to general graphs with cycles has been slow. (Also, it is not 
clear that minimal realizations are necessarily the best ones 
for iterative decoding; see Section HV-HI ) 

A fundamental issue is that in general there is no unique 
minimal realization of a linear code or system on a graph 
that contains cycles. As was observed by Koetter and Vardy 
ifTsl l. even for the simplest such graph comprising a single 
cycle (called a "tail-biting trellis realization"), there are various 
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notions of minimality, and for any such notion there is only 
a partial ordering of tail-biting trellis realizations. However, 
Koetter and Vardy were able to specify a small, tractable set 
of possibly minimal tail-biting trellises, by showing that every 
linear tail-biting trellis realization factors into a product of one- 
dimensional "atomic" realizations ifTTl (as do conventional 
linear trellis realizations 1*191). 

On more general graphs, there is no unique minimal real- 
ization, and also no such product factorization (see Appendix 
A). How then can we pursue something like minimality? 

The main idea of this paper is to look for possibilities 
for "locally" reducing the complexity of a realization — e.g., 
by reducing the size of one internal variable alphabet ("state 
space") — without changing the code that is realized or the 
complexity of the rest of the realization. Such an approach 
follows a long tradition in the conventional trellis minimization 
literature (e.g., Il20ll . ||25| ). and was previously explored by 
Koetter 1161 . In Appendix B, we give a brief summary of 
Koetter' s pioneering work in the language of this paper 

For instance, a realization in which some state variable 
value never occurs is obviously locally reducible, as the state 
alphabet may be trimmed to the set of values that actually 
occur But as Koetter |16| observed, duality considerations 
using the "dual realizations" of iU can reveal less obvious 
dual local reducibility criteria. For example, we give a simple 
proof that a constraint code is not (state-)trim if and only if 
the dual constraint code is not "proper," in a sense that extends 
the traditional definition of "properness" for trellis realizations 
|25|. By duality, any realization that involves an "improper" 
constraint code must therefore be locally reducible. 

We next show that a finite linear or group normal realization 
on a cycle-free graph is minimal if and only if every constraint 
code is both trim and proper Although this simple result has 
long been known for conventional trellis realizations ||251 . it 
seems to be new for realizations on more general cycle-free 
graphs. This development yields a constructive proof of the 
fundamental State Space Theorem, and leads immediately to 
straightforward iterative minimization algorithms. 

Next, we consider linear or group realizations on general 
graphs that are not one-to-one — i.e., every codeword in C 
is realized by multiple configurations. Such realizations are 
called unobservable, since the values of the internal variables 
are not determined by those of the external variables. Such re- 
alizations would seem to be obviously undesirable for iterative 
decoding; see Section HV-HI 

We show that any unobservable realization is locally re- 
ducible. Therefore we may always assume that any linear or 
group realization on a general graph is observable. 



Following the principle that the dual of a locally reducible 
normal realization of C is locally reducible, we show that 
the dual of an unobservable linear realization is a realization 
with dependent constraints. For consistency with classical 
terminology, we call such a realization uncontrollable. By 
duality, an uncontrollable realization is locally reducible. 

However, it is not so clear that an uncontrollable realization 
is unsuitable for iterative decoding; see Section IIV-HI For 
example, we show that any parity-check realization with 
redundant parity checks is uncontrollable; nevertheless, LDPC 
codes with redundant checks have sometimes been preferred 
in practice. 

We show that an uncontrollable tail-biting trellis realization 
consists of disjoint subrealizations, as with classical uncontrol- 
lable conventional trellis realizations. However, realizations on 
general graphs do not in general have this property. 

Finally, given an unobservable realization on a general 
graph, the support of the unobservable configurations must 
be a cycle or generalized cycle (a subgraph whose vertices all 
have degree 2 or more). In the dual uncontrollable realization, 
there exist nonlocal constraints that are defined on the same 
support. In the binary case, this support must actually be an 
Eulerian cycle (a generalized cycle whose vertices all have 
even degree). 

We may summarize our progress on establishing conditions 
for local reducibility as follows. We partition the universe of 
linear or group codes on finite graphs into four quadrants, as 
shown in the 2x2 matrix below, where one axis distinguishes 
trellis from non-trellis graphs, and one cycle-free from cyclic. 





trellis 


non-trellis 


cycle-free 


conventional trellises 


cycle-free graphs 


cyclic 


tail-biting trellises 


general graphs 



For conventional trellis realizations, simple conditions for 
minimality have long been known. In this paper, we find a sim- 
ple, conclusive, and locally testable condition for minimality 
for a general finite linear or group cycle-free realization: 

minimal <^ every constraint code is trim and proper 

For linear tail-biting trellis realizations, which in general 
have no unique minimal realization, we make some progress 
in this paper toward finding conditions for local irreducibility. 
In a subsequent paper (TQ\, we will give necessary and 
sufficient conditions for a linear tail-biting trellis realization 
to be irreducible, under a more refined definition of local 
reducibility. These results largely settle the tail-biting trellis 
case, although we consider that there are still some open issues 
in this case. 

Finally, for the general case, beyond trimness and proper- 
ness, the dual concepts of observability and controllability 
yield further local reducibility criteria; however, this case 
remains largely open. 



II. Codes, realizations, and duality 

In this paper we will focus on linear codes over a finite field 
F. Everything generalizes to group codes over finite abelian 
groups, and to linear systems over the real or complex field. 
We will state some of our most basic results for both the linear 
and group cases, since the linear case is often more familiar, 
whereas the group-theoretic statements and proofs are often 
more transparent. 

A. Codes and realizations 

A linear code C over a finite field F is a linear subspace 
of a symbol configuration space A = Ilkei^Ak, where each 
symbol alphabet Ak is a finite-dimensional vector space over 
F, and 2^ is a discrete index set. We will usually assume 
that 2^ is finite (i.e., C is a block code); then the symbol 
configuration space A is finite. We may alternatively refer to 
a symbol variable as an external variable Ak, which takes 
values Ofe in an external variable alphabet Ak- 

Similarly, a group code C is a subgroup of a symbol 
configuration space A — HkeiA-^k, where each symbol 
alphabet Ak is a finite abelian group. If I4 is finite, then 
1^1 = Ilfcei^lAI is finite. 

For a state realization of C, we define also a set {Sj : 
j E Is} of state spaces Sj indexed by a state index set I5, 
and a set {Ci : i £ Xc} of local constraint codes Ci indexed 
by a constraint index set Ic, where each constraint code Ci 
involves subsets of the symbol and state variables indexed by 
2^^(o C I4 and Iga) C I5, so d C ^(*) x S^^\ where 



-^^'^=nfeei.„..A. and5W=n, 



Sr 



In a linear realization, each state space Sj and each con- 
straint code Ci is a vector space over F; in a group realization, 
each state space and constraint code is a finite abelian group. 

The state configuration space is 5 = Hj^i^Sj. We may 
alternatively refer to a state variable as an internal variable 
Sj, which takes values Sj in an internal variable alphabet Sj. 

The degree of a variable is the number of constraint codes 
in which it is involved; the degree of a constraint code is the 
number of variables that it involves. 

A configuration (or trajectory) of the realization is a pair 
(a, s) G A y. S for which all constraints are satisfied; i.e., 
(aW,s(''^) e Cj,Vi G Ic- Its full behavior is the set <B 
of all such vaUd configurations. The code C realized by the 
realization is then the set of all a G ^ that appear in some 
(a, s) G S; i.e., C is the projection of *8 onto A. A linear 
realization realizes a linear code; a group realization realizes 
a group code. 

B. Observability 

A state realization is called one-to-one, or observable, if 
there is precisely one pair (a, s) G 03 corresponding to each 
aGC. 

For a linear or group realization, we define the unobservable 
state configuration space S^ as the subspace of unobservable 
state configurations s G 5 such that (0,s) G *B; thus such a 
realization is observable if and only if 5" is trivial. Indeed, 
iS" is evidently isomorphic to the kernel /C" — {(0, s) G 



S} of the projection of *B onto A, whose image is C, so 
by the fundamental theorem of homomorphisms C = S//C". 
Therefore \C\ = |*B|/|iS"|, or, in the linear case, dimC — 
dim*8 — dim 5". 

C. Minimality and local reducibility 

As in ifTSl . a realization with state spaces Sj will be called 
minimal if there exists no realization of the same code with 
state spaces Sj that has the same graph topology {i.e., the same 
index sets 14,25, Ici^^io, Is(i)) such that \Sj\ < \Sj\ for 
all j, with at least one strict inequality. 

A realization of a code C with state spaces Sj and constraint 
codes d will be called locally reducible if it is possible to 
replace a single state space Sj by a strictly smaller space, and 
the constraint codes involving Sj by constraint codes that are 
no bigger, without changing the rest of the realization or the 
code that it realizes. Evidently a locally reducible realization 
is nonminimal. 

Our basic tools for local reduction will be trimming (restrict- 
ing) and merging (taking quotients) of state spaces, which we 
will see are dual operations. 

D. State variables of degree one 

As a first exercise in local reducibility, we observe that any 
realization that includes a nontrivial state variable of degree 1 
is locally reducible, since we may always reduce the alphabet 
of that variable to the trivial alphabet (i.e., merge all states 
with the zero state) without changing the code realized by 
the realization, there being no other constraints on that state 
variable. Thus we will assume henceforth that state variables 
of degree 1 have a trivial state space. 

We note further that, in any state realization, a state variable 
whose alphabet size is one may optionally be deleted from 
the realization without affecting the code that is realized. 
However, this is not the usual convention for end states in 
trellis realizations; see below. 

E. Normal realizations 

A state realization is called normal if the degree of every 
symbol variable is 1, and the degree of every state variable is 2. 
(More generally, a state variable could have degree 1, provided 
that its alphabet is trivial; see the previous subsection.) 

As shown in 0|, any realization may be straightforwardly 
"normalized" by introducing replica internal variables and 
equality constraints, as follows. For every constraint code Ci 
and every variable Ak or Sj involved in Ci, create a replica 
internal variable Aki or Sji\ then, for every Sj, introduce an 
equality constraint on all the replicas Sji of Sj and delete the 
original variable Sj, and for every Ak, introduce an equality 
constraint on all the replicas Aki of Ak, plus the original 
external variable Ak- The resulting realization is normal, 
realizes the same code C, and has essentially the same graph 
topology. Thus we will assume henceforth without essential 
loss of generality that all realizations are normal. 

A normal realization has a natural graphical representation, 
called a normal graph, in which constraints are represented by 



vertices, internal variables by ordinary edges (edges of degree 
2), and external variables by half-edges (edges of degree 
1). An edge or half-edge is incident on a vertex whenever 
the corresponding variable is involved in the corresponding 
constraint code. 

If the graph of a realization is disconnected, then the behav- 
ior and the code that it realizes are the Cartesian products of 
the respective behaviors and codes realized by the components. 
We will therefore assume that the graphs of all realizations are 
connected. 

F. Trellis realizations 

A trellis realization is a state realization in which every 
constraint code involves precisely two state variables, every 
state variable is involved in one or two constraint codes and 
every symbol variable is involved in one constraint code. Thus 
the graph of a trellis realization must be one of the following 
(if connected): 

> a finite, semi-infinite or bi-infinite chain graph (called a 
"path" in graph theory), which we call a conventional 
trellis realization; 

• a single-cycle graph, which we call a tail-biting trellis 
realization. 

Thus bi-infinite and tail-biting trellis realizations are inherently 
normal. Finite or semi-infinite conventional trellis realizations 
are not strictly normal, because they have "end" state variables 
of degree 1, but they may be taken to be normal if the 
associated degree- 1 state alphabets are trivial. 

We note that a finite conventional trellis realization may be 
considered to be a special case of a tail-biting trellis realization 
in which one state variable has a trivial alphabet, and therefore 
could be deleted. 

We will sometimes depict a trellis realization by a traditional 
trellis diagram, in which all branches, states and symbols are 
depicted explicitly. 

G. Duality 

We now briefly recapitulate some of the basic duality 
principles for both the group and linear cases that will be 
used heavily in this paper, following |9|. 

If G is a finite abelian group, then its dual group G is the 
character group of G, namely the set of all homomorphisms 
from G to the circle group M/Z. The dual group of G may be 
taken as G. In the finite abelian case, G and G are isomorphic. 

For g G G and g ^ G, there is a well-defined pairing 
{g,g) ~ g{g) G M./Z that has the usual bihomomorphic prop- 
erties: e.g., (0,g) = (.9,0) = 0, (.91 +52,. 9) = {91, 9) + {92, 9), 
and so forth. 

Elements g E G and g E G are said to be orthogonal 
if (.9i.9) = 0. If iJ is a subgroup of G, then the orthogonal 
subgroup H^ is the set of all elements of G that are orthogonal 
to all elements of H. H^ is indeed a subgroup of G, and the 
orthogonal subgroup to H^ is H. 

The dual group H to H C G is isomorphic to the 
quotient group G/ H^. In the finite abelian case, this implies 
\H\\H^ = \G\ = \G\. 



aeA = UkAk 



n,c, 



s e 5 = HjSj 
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Fig. 1. Normal realization duality: Dual realizations realize oithogonal codes. 
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s £ 5 = n^Sj 
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Similarly, if G is a finite-dimensional vector space over 
a finite field F, then it has a dual space G, a vector space 
over F of the same dimension, which is furnished with 
an inner product {g,g) G F that is a bilinear form: i.e., 
(0,g) == (.9,0) = 0, (.91+32,3) = (.91,5) + (52,. 9), and so 
forth. It is well known that all dual spaces of G are isomorphic, 
where the isomorphism preserves the inner product. If G = F^ 
for some i, then we may and will choose G — F^, along with 
the standard inner (dot) product on F^. 

Moreover, if {61, . . . , 6„} is a basis for an n-dimensional 
vector space G, then there exists a dual basis {&!,...,&„} 
for G such that {bi,bj) = Sij, the Kronecker delta function. 
Given any g G G, there exists a unique coordinate vector 
a G F" such that g ~ J2j '^j^j'^ '^his establishes an iso- 
morphism between G and F". Similarly, each g ^ G has a 
unique coordinate vector a G F" such that g = J^i'^i^i- ^^ 



J2j ctjbj and g = J2iaibi, then 
{9,9) =X15I"»"J^^»'^J'^ =Y1 



aiUj = a ■ a: 



i.e., the inner product {g,g) is the dot product a ■ a of the 
coordinate vectors. In other words, dual bases define adjoint 
isomorphisms G O F" and G -f-?" F" that preserve inner 
products. 

As with groups, if i7 is a subspace of G, then the orthogonal 
subspace H^ is a subspace of G, and the orthogonal subspace 
to H^ is H. Moreover, dim H + dim H^ = dim G = dim G. 

If Q = HkGk is a finite direct product of a collection of 
groups or vector spaces Gk, then the dual group or space to 
Q is the finite direct product Q = UkGk, and the pairing or 
inner product between g G ^ and g G ^ is given by the 
componentwise sum 



(g,g) ^^{Sk,gk)- 



If H = TlkHk is a direct product of subgroups or subspaces 
Hk Q Gk, then the orthogonal space is the direct product 

u^ = iikHt c g. 

H. Projection/cross-section duality 

Projection/cross-section duality is one of the most funda- 
mental and useful duality relationships for linear and group 
codes [4|. 

Given a linear or group code C C A = Jlfcei -^k, let ■J Q 
I_4 be any subset of the symbol index set, and /C = I4 — J' 
its complement. Then A may be written as the direct product 
A'^ X A'^, where A'^ = flkej-^k and A'^ = Ukeic-^k- 
Correspondingly we may write any a G ^ as a pair (a-^, a'^), 
where a-^ G A'-^ and aJ^ G A'^ are the projections of a on ^7 
and /C, respectively. 



The projection map Pj : A — s> A'-^ defined by a M> a'^ 
is evidently a homomorphism. Given a linear or group code 
C C A, the projection of C onto A'-^ is C^j^j = Pj{C), a 
subgroup of A'-^ . 

The cross-section C.j^j of C on A'^ is defined as C._4J = 
{a-^ G A'^ I (a-^,0'^) G C}, which is evidently a subgroup 
of Ci^j. We note that C.^.7 is trivially isomorphic to the 
kernel of the projection map P/c : C — > A'^, a subcode of 
C; therefore the following duality relationship is sometimes 
called projection/subcode duality. 

Lemma (Projection/cross-section duality). If C C A-^ x A'^ 

and C^ C A'-^ x A'^ are orthogonal linear or group block 
codes, then C.^j and {C^)i^j are orthogonal linear or group 
codes. 

Proof. Because pairings or inner products are component- 
wise sums, we have (a, a) = {k^ ,sc^) + {k^,SL'^). Thus 
(a'^,a''') _L (a'^,0'^) <;=> a-^ ± a-^. We therefore have the 
following logical chain: 



a^ G C.,AJ ^ {a-' ,0'^) eC^ (a^ , O'^) ± C^ O a^ _L (C^)|^.7. 

/. Dual realizations 

The dual realization of a normal linear or group realization 
is defined below. The normal graph duality theorem of [4] 
states that the dual realization realizes the orthogonal code 
C^ to C. By now there exist several proofs of this theorem, 
the most elegant of which involve Fourier transforms of 
code indicator functions ll22l . [l], Q. We give here a new 
elementary proof (similar to that of Koetter [16|), partly in 
order to introduce concepts and notation that will be useful 
later in Section HV] 

Consider the set HiCi of all possible combinations of 
codewords of the constraint codes C,. With a normal real- 
ization, every external variable Ak is involved in precisely 
one constraint code, and every internal variable Sj is involved 
in precisely two constraint codes. Let us replace Sj by a 
replica variable S*' in one of its two appearances (it does not 
matter which), and add an equality constraint s' = Sj. Then 
each element of Il^Ci may be uniquely identified by a triple 
(a, s,s') G Ax S X S. 

A valid configuration (a, s,s') G IliCi is one for which 
s' = s. Thus the behavior of the realization is *8 = {(a, s) G 
^ X 5 I (a, s, s) G fliCi}. This is illustrated in Fig. [Tfa)- 

The dual realization is defined as the realization with the 
same graph topology in which the external alphabets Ak are 
replaced by their duals Ak, the internal alphabets Sj are 
replaced by their duals Sj, the constraint codes Ci are replaced 
by their orthogonal codes C:^, and each equality constraint 
s'j = Sj is replaced by the sign inversion constraint s' = —Sj, 
as illustrated in Fig. [Ub). Thus the dual behavior may be 



written as <B° == {(a, s) e Ax S \ (a,s, -s) e U,C:^}- 
Note the slight sign asymmetry between the definitions of the 
primal and dual behaviors. 

Alternatively, a dual way of defining the primal behavior 
is as follows. Following Koetter fT6\, we may write *8 = 
{(a, s) e y^ X 5 I (a, s,s) _L 11^0^^}; that is, <B is the set 
of all configurations (a, s) such that (a, s,s) _L (a, s,s') for 
all (a, s,s') G HiC^, or equivalently (a, s) _L (a, s + s'). 
Therefore if we define *8^ as the image of the homomorphism 
E : Yii ^t -^ A y. S that is defined by the sum map 
(a, s, s') H- > (a, s + s'), then *8 is the orthogonal code to *8^. 

Notice now that (a, s, s') £ Y\.i ^t corresponds to a valid 
configuration (a, s, — s) in the dual behavior if and only if it is 
mapped by S to (a, 0). Therefore the code V realized by the 
dual behavior is precisely the cross-section (*B^)._4 = {a £ 
A I (a, 0) £ *B^}. But by projection/cross-section duality, the 
orthogonal code to V must be the projection *B|^, which is 
C; i.e., V = C^. 

In summary, we have proved: 

Lemma (Normal realization duality). If a normal realization 
as in Fig. [Tta) realizes a linear or group code C, then its dual 
realization as in Fig. [Tib) reaUzes the orthogonal code C^. 

U 

III. Trimness and properness 

In this section we discuss trimness and properness of 
constraint codes. These are dual properties; i.e., Ci is trim 
if and only if Cj^ is proper. We show that a realization is 
locally reducible if it has a constraint code that is not both 
trim and proper. Finally, we show that a linear finite cycle- 
free realization is minimal if and only if all constraint codes 
are trim and proper. 

A. Trim-proper duality 

A constraint code Ci will be called trim if the projection of 
Ci onto every state space Sj that is involved in Ci is Sj; i.e., 
if every such projection is surjective. 

A constraint code Ci will be called proper if the value of any 
state variable Sj involved in Ci is determined by the values of 
all other variables; i.e., if for any set of values of all variables 
involved in Ci other than Sj, at most one codeword in Ci has 
those values. In the linear or group case, Ci is proper if and 
only if there is no codeword of Ci whose support is a single 
state variable Sj; i.e., for all Sj the cross-section {Ci);s- is 
trivial. 

This definition of "proper" generalizes the traditional def- 
inition for a trellis realization, which is called proper [E3l 
(or "biproper" f25l) if the symbol values associated with all 
transitions to or from a given state are all differently 

As in lfT2l . we observe that trimness and properness are dual 
properties: 

Theorem 1 (Trim-proper duality). A linear or group con- 
straint code Ci is trim if and only if its orthogonal code C,^ is 
proper. 

'in system theory, for conventional state-space (trellis) realizations, proper- 
ness is sometimes called "instantaneous observability," because then and only 
then is the next state determined by the current state and symbol. 



Proof: By projection/cross-section duality, the cross-section 
(C,^).^ is the orthogonal code to the projection {Ci)\s- Thus 
{0} C \ct).s^ if and only if (C,)|5, C 5^0 ' D 

This duality is illustrated in Fig. |2] Ci is not trim at Sj 
if and only if the projection {Ci)\s. is a proper subspace or 
subgroup T C Sj. In this case Ci may be represented as the 
concatenation of a trimmed constraint code Ci, in which Sj 
is replaced by T without changing any codewords, and an 
inclusion constraint code C^-j. = {(f, t) £ T x iSj : f £ T}, as 
illustrated in Fig. 121 a). 



(C. 
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{A. 



{c^y 
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(a) 



(b) 



Fig. 2. Dual realizations: (a) nontrim d; (b) improper dual {d)^ . 

By definition, the dual realization of the improper code 
{Ci)^ is obtained by concatenating the orthogonal code (C^)^, 
a sign inverter, and the orthogonal code (C^)^. It is easy to 
see that {C^)^ ^ {{-s + T^, s) £ Sj/T^ x Sj : s e Sj}, 
using the inner product {s + T^,t) — (s,i) for i £ T 
and s £ SjU Combining the sign inverter and (C^)^, we 
obtain the quotient constraint code C^ = {(* + T^^s) £ 
Sj/T'^ X Sj : s ^ Sj}, which involves the natural map from 
Sj to its quotient Sj/T^, as shown in Fig. |2b). 

In view of Theorem 1 , every improper constraint code may 
be represented as in Fig. |2lb) as a code with a reduced state 
space Sj = Sj/T^, obtained by merging all states in each 
coset s + T^, plus a constraint code which allows every coset 
of T^ to branch to all of its elements. 



B. Local irreducibility requires trimness and properness 

We now show that both "trim" and "proper" are necessary 
for local irreducibility. 

Theorem 2 (Local reducibility). A normal linear or group 
realization is locally reducible if any constraint code Ci is not 
both trim and proper. 

Proof: We give a pictorial proof, following Fig. |2] Suppose Ci 
is not trim at Sj; i.e., T — {Ci)\s- C Sj. Let C^/ be the other 
constraint in which Sj is involved. Then, as shown in Fig. Oa), 
the combination of C,; with Ci' is equivalent to the combination 
of the trimmed code Ci with a second trimmed code Ci', 
namely the combination of Ci' with C^, which restricts Sj 
to T. Replacing the combination of Ci and Ci' with that of Ci 
and d' thus strictly reduces the connecting state space from Sj 
to T, without changing the code C realized by the realization. 
Dually, suppose d is not proper at Sj; i.e., {0} C T = 
{Ci);s-. Again, let Ci' be the other constraint in which Sj is 
involved. Then, as shown in Fig. Ob), the combination of Ci 
with Ci' is equivalent to the combination of the merged code 

-By "A C -B," we mean that yl is a proper subset of B. 
^In other words, the inclusion map 7} "-^ Sj and the natural map Sj — f 
Sj/{Tj)'^ are adjoint homomoiphisms. 
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Fig. 3. Dual reductions: (a) nontrim d; (b) improper d. 
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d with a second merged code d', namely the combination of 
Ci' with C<_, which merges Sj to Sj/T via the natural map. 
Replacing the combination of Ci and Ci' with that of d and 
Ci' thus strictly reduces the connecting state space from Sj to 
Sj/T, without changing the code C realized by the realization. 

D 

Theorem 2 and its proof show that in any normal linear 
or group realization, if any constraint code is not trim or 
improper, then there is a strict reduction of the corresponding 
state space Sj via trimming or merging, respectively, such 
that only the two constraint codes in which Sj is involved 
are affected. It is therefore hard to imagine any application 
in which one would not begin by ensuring that all constraint 
codes have been made trim and proper 

This development shows that "merging" (quotient-taking) is 
the dual reduction to "trimming" (restricting). Quotient- taking 
preserves the linear or group property of the realization, and 
thus is the appropriate notion of "merging" for Unear or group 
realizations. 

For conventional and tail-biting trellis realizations, the fact 
that "improper" implies "mergeable" is well known; see, e.g., 

m. 

Example 1 (compare f25', Fig. 2]). Consider the binary linear 
block code C — {000, 110}. This code may be realized by the 
linear trellis realization shown in Fig. SJa), with three binary 
symbol alphabets, four state spaces Sq — S^ — {0},Si — 
S2 ~ {0, 1}, and three constraint codes Cq = {000, Oil}, Ci = 
{000, 111}, and C2 = {000, 100}. Since C2 has a nonzero 
codeword 100 supported by the single state space S2, C2 is 
improper, and therefore can be reduced. Indeed, the two states 
of S2 can be merged to a single state {0} without affecting C; 
this yields the minimal linear trellis realization of this code, 
shown in Fig. SJb). 
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Fig. 4. (a) Improper, (b) merged, (c) non-trim and (d) trimmed realizations 
of dual codes. 

The dual linear trellis realization to that of Fig. HJa) uses 
the same variable alphabets and the orthogonal constraint 
codes C(f = {000, Oil}, Cj^ = {000,011,110,101}, and 
C2 = {000, 010}, as shown in Fig. SJc). (Because the ground 
field is binary, no sign inverters are needed.) It can be seen 



that this dual realization, although unusual, does indeed realize 
C^ = {000, 110,001, 111}. Since the projection of C^ onto 
S2 is {0}, C2 is not trim, and therefore can be reduced. Indeed, 
restricting 52 to {0}, we obtain the minimal trellis realization 
of C-L of Fig. Etd). D 



C. Minimal cycle-free realizations 

It is well known that a finite conventional linear trellis 
realization is minimal if and only if all end state spaces are 
trivial and every constraint code Ci is trim and proper ["251. We 
now show that this result holds for any finite cycle-free linear 
or group realization. As a byproduct, we obtain a constructive 
proof of the State Space Theorem, which is perhaps the most 
fundamental result of linear behavioral system theory [27 1. 

The key graph-theoretic property of cycle-free graphs is 
that every edge is a cut set. In other words, if Sj is any 
state space in a normal cycle-free realization, then cutting the 
edge corresponding to Sj partitions the normal graph of the 
realization into two disconnected subgraphs, which we label 
arbitrarily as Vj and Tj (for "past" and "future"), as shown 
in Fig. ID 

Furthermore, if we define the past depth (ff of the edge 
Sj as the maximum distance to any leaf node in Vj, then the 
depths of all other states involved in the constraint code T>^ 
in Vj to which Sj is connected must be less than d^ (see Fig. 
nil. (Again, we assume there are no degree-1 states.) 

We correspondingly represent the symbol configuration 
space A as the product A'^^ x yl-^J of past and future 
components, and write a symbol configuration as a pair 
a = {st^^ , a-^j). The projection and cross-section of the code 
C on the past are written as C,^Vj and C.^v^ , respectively. We 
similarly define past and future state configurations s^^ £ S^^ 
and s-^^ G iS-^J, where Sj is included in both S^^ and S''^^ . 
Finally, we say that a past configuration a^^ reaches a state 
Sj £ Sj if there is a past configuration {sl^j , s^j ) that satisfies 
all past constraints and includes Sj. 

We then have the following remarkably simple result: 

Theorem 3 (Minimal = trim + proper). Given a normal 
linear or group realization of a code C on a finite connected 
cycle-free graph Q, the following are equivalent: 

(1) The realization is minimal. 

(2) Every constraint code C,; is both trim and proper 

(3) Every state space Sj is isomorphic to the "past-induced 
state space" C^^Vj /C.^^p^, and the set of a^^ that reach 
a given state Sj e Sj is the corresponding coset of C.^Vj 
in C^^-P, . 
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Fig. 5. Illustration of the past Vj and future Tj of an edge Sj 



Proof: 

(1 => 2) By Theorem 2, if any Ci is not both trim and proper, 
then the reaHzation is locally reducible, and thus not minimal. 

(2 ^ 3) Let {C'^^)ij(Pj denote the set of all past configura- 
tions sJ^^ G A'^^ that appear in some valid past configuration 
(a^'j, s^'j). We prove by finite induction on d^ that 

(a) trim =^ every state Sj G Sj is reached by some symbol 
configuration sP^ e {C'^^).^vj; 

(b) proper => every symbol configuration a^^ G i'^'^^)\A'^i 
reaches a unique state Sj G Sj. 

In other words, a trim and proper realization realizes a well- 
defined surjective "reaching map" R : {C'^^)\XPj -^ Sj. 

Clearly (a) and (b) hold if the depth of 6^ is 1; i.e., if Vj 
comprises a single leaf node, representing a constraint that 
involves no state other than Sj. Now if (a) and (b) hold for 
all depth-(< dj) edges in Vj, then (a) and (b) hold for all 
depth-d'^ edges Sj, because, denoting the constraint code to 
which Sj is connected in Vj as V^ (see Fig. |5]i: 

(a) if V^ is trim, and all of its depth- (< d''^) states are 
reachable, then all states Sj G Sj are reachable, since Sj 
occurs with some configuration of depth- (< d''^) state 
values in some codeword of V^; 

(b) if Vj is proper, and every symbol configuration in the 
"past" of its depth-(< d^) states reaches a unique state 
value, then the same must be true for Sj, since otherwise 
there would be two codewords of Pj that differ only 
on Sj. 

In particular, the all-zero symbol configuration 0^^ reaches 
only the zero state of Sj. 

Now, applying the same argument to Fj, we conclude that 
all states of Sj are reached by future configurations, so all 
valid past configurations are past projections of codewords; 
i.e., {C'^^).^Vj = C,j^Vj. Moreover, the unique state of Sj 
that can be reached by the all-zero future symbol configuration 
O'^^ is the zero state. Hence the set of configurations in C^^Vj 
that reach the zero state in Sj is precisely the cross-section 
{a^^GC|_4., |(a^^o^OeC}. 



^■.A^o 



By linearity, the "reaching map" R : C, 



A' i 



Sj is a 



homomorphism. We have shown that it is surjective and has 
kernel C.^p^ . Thus, by the fundamental theorem of homo- 
morphisms, Sj = C,jVj jC.j^^ . Moreover, since the subset of 
configurations in Ci^p^ that reach the zero state in Sj is C.jt^ , 
by linearity the subset of configurations in C^j^j that reach 
an arbitrary state in Sj is the corresponding coset of C.j^j . 

(3 => 1) Since (3) is equally valid for Tj and Vj, it follows 
that Sj is also isomorphic to the "future-induced state space" 
Ci^^j /C._^^j, and the set of a.^^ G C|_^^j that reach a given 
state Sj G Sj is the corresponding coset of C.j^r, in Ci^^j. 



Thus if {x^^isj') : Sj G Sj} is a set of coset repre- 
sentatives for the cosets of C.j^Vj in C^j^j, and {r-^^^Sj) : 
Sj G Sj] is a set of coset representatives for the cosets of 
C.j^Tj in Csj^Tj, then the set of all symbol configurations 
(a^J, a-^^) G Jp^ X A^^ that pass through a given state 
Sj G Sj is (r^j(sj),r-^j(sj)) + C._^Pj x C._^^j, a coset of 
C.j^j X C.j^Tj . The code C is thus the union of these \Sj \ 
disjoint cosets. (An easy corollary is that Sj is also isomorphic 
to C/(C_4P, xC^^,).) 

This shows that every realization of C with the same graph 
topology must have at least 15^1 states at time j, because 
(a^j, a-^j) G J^^ X A'^' is in C if and only if a^^ and 
a-^' are in corresponding cosets of C.^Vj and C.j^fj . Thus 
the realization is minimal. D 

This proof shows constructively that in a minimal cycle- 
free realization there is a one-to-one match of the cosets of 
C.^j in C|_^Pj with the cosets of C.j^tj in Ci^j^^ such that 
every a^^ in a given coset can be followed by every a-^^ in its 
matching coset to form a codeword in C, and no other pairs 
(a'^j , a-^3 ) are codewords in C. This is the essence of the State 
Space Theorem ll27l . ||8], iHJ. This theorem also yields the SST 
isomorphisms 






^\A^^ 



^■.A^i ^ ^-.A^^ 



and the fact that a minimal state space Sj is isomorphic to 
any of these quotients. 

In general, we will say that a realization is trim (resp. 
proper) if every constraint code is trim (resp. proper). We will 
further define a reaHzation to be state-trim if the projection of 
the full behavior *8 onto every state space Sj is surjective 
(i.e., equal to Sj), and branch-trim if the projection of 58 
onto every constraint code Ci is surjective. Thus "trim" and 
"proper" are local properties, since they involve only local 
constraint codes, whereas "state-trim" and "branch-trim" are 
global, since they involve the full behavior Notice that the 
second part of this proof shows that for a finite cycle-free 
realization, "trim" implies "state-trim" and "branch-trim." 

Koetter |16 Lemma 6] shows that a state space Sj may be 
reduced by merging if the dual behavior 58° is not state-trim at 
Sj, a global condition that yields our local result "improper" 
=> "locally reducible by merging" as a corollary. 

Kashyap flSJI has shown that a given tree realization may be 
minimized by first making it state-trim ("essential"), and then 
merging each state space Sj into its quotient Sj/Wj, where 
Wj is the subspace of Sj that is reached by configurations in 
the zero-state subcode C.^v^ x C.^:fj . However, because the 



latter step requires computing the full behavior, this procedure 
is not very efficient. 

From Theorem 3, it follows that if a linear or group 
realization on a finite cycle-free graph is nonminimal, then it 
may be made minimal by a finite sequence of local reductions. 
In other words, there exists a straightforward and efficient 
finite algorithm for minimizing any given linear or group 
realization on any finite cycle-free graph. 

From Theorems 1 and 3, it follows that the dual to a finite 
cycle-free minimal realization is minimal. The dual minimal 
state spaces are thus the dual groups or spaces to the primal 
minimal state spaces; i.e., we obtain the dual state space 
theorem |9] as a corollary. 

Finally, we remark that the "shortest basis" approach to 
minimality that is often used for conventional linear trellis 
realizations (see ||6l and references therein) cannot be extended 
to general cycle-free realizations, because it relies on the 
"product factorization," which generally does not exist for 
general cycle-free graphs; see Appendix A. In this respect, the 
"trim and proper" approach to minimality may be regarded as 
more basic than the "shortest basis" approach. 

IV. Observability and controllability 

In this section, we define a realization to be controllable 
if its constraints are independent, and give a simple test for 
controllability. We show that a realization is uncontrollable if 
and only if the dual realization is unobservable, and that in 
either case such a realization is locally reducible. 

We give conditions for controllability and observability 
for finite cycle-free realizations, generator realizations, parity- 
check realizations and tail-biting trellis realizations. A tail- 
biting trellis realization is uncontrollable if and only if its 
behavior consists of disconnected subbehaviors, as with clas- 
sical uncontrollable conventional trellis realizations; however, 
on more general graphs, uncontrollability does not necessarily 
imply disconnectedness. We show that the support of an 
unobservable configuration must be a cycle or a generalized 
cycle. 

We conclude by observing that for iterative decoding, 
unobservable realizations seem clearly undesirable; however, 
uncontrollability may not hurt, and may even be advantageous. 

A. Independent constraints and controllability 

The behavior *B of a linear realization is defined by a system 
of Unear homogeneous constraint equations. Specifically, each 
constraint code C^, i G Ic of the realization may be specified 
by dim C:^ independent equations, so 03 is the solution space 
of X^iGi dimCj^ homogeneous equations. We say that the 
realization has independent constraints if the system of all 
these equations is linearly independent. 

We then have the following fundamental duality theorem: 

Theorem 4 (Observability/controllability duality). A normal 
linear or group realization has independent constraints if and 
only if its dual realization is observable. 

Proof: Suppose the system of constraints is hnearly de- 
pendent; i.e., there exists a nontrivial linear combination 



X;,(aW,sW) = (0,0) with (a(^),sW) e (Ci)-^ for alH. Since 
each symbol variable is involved in exactly one constraint 
code, this implies a^. — for all k. Moreover, since each 
state variable is involved in exactly two constraint codes, 
this implies that the two corresponding values Sj and s' 
satisfy Sj = — s'. But all this implies that (0, s) is a valid 
configuration in the dual realization (see Fig. Htb)). Thus 
the dual realization is unobservable. The converse follows by 
reversing these arguments. D 

For classical conventional state (trellis) realizations, the 
dual property to observability is called controllability. We will 
therefore call a linear or group realization that has independent 
constraints controllable. We use this term even though (a) 
the classical definition arose at a time when linear system 
theory was embedded in control theory, which is not our 
context here; (b) although the behavior of uncontrollable tail- 
biting trellis realizations is similar to that of uncontrollable 
conventional trellis realizations, as we will see in Theorem 10, 
such uncontrollability properties do not necessarily extend to 
realizations on general graphs, as we will see in Section IIV-GI 
The reader who is not so interested in continuity with classical 
linear system theory may therefore prefer terms like "one-to- 
one" and "independent" to "observable" and "controllable." 

Example 2 {cf. |18, Fig. 5]). The binary linear (3,2,2) 
block code C = {000,110,101,011} may be realized by 
the linear tail-biting trellis realization shown in Fig. |6ja), 
with three binary symbol alphabets, three binary state spaces 
So = Si = S2 = {0, 1}, and three constraint codes Co = Ci 
= C2 = {000,110,101,011}, where C2 involves 52 and Sq. 
Because the all-zero codeword is realized by two configu- 
rations, this realization is unobservable, with dimiS" = 1. 
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Fig. 6. (a) Unobservable and (b) uncontrollable tail-biting realizations of 
orthogonal codes. 

The dual linear tail-biting trellis realization to that of Fig. 
|6la) uses the same variable alphabets and the orthogonal 
constraint codes Cjf = C^ = C2 = {000, 111}, shown in Fig. 
|6tb). (Again, because the field is binary, no sign inverters are 
needed.) The dual realization realizes the orthogonal (3, 1, 3) 
code C^ ~ {000, 111}; however, since it is the dual to 
an unobservable realization, it is uncontrollable by Theorem 
4. Explicitly, the three constraint equations corresponding 
to the branches 101 in the three primal constraint codes 
{i.e., the constraints that form the unobservable configuration 
(a, s) = (0, 1) in the primal behavior) are dependent. D 

B. Finite cycle-free realizations 

For the finite cycle-free realizations that were considered in 
Section UlI-CI we have immediately: 

Theorem 5 (Finite cycle-free realizations). A normal Unear 
or group realization on a finite cycle-free graph is observable 
if it is proper, and controllable if it is trim. 



Proof. From the second part of the proof of Theorem 3, part 
(b), if a reaHzation on a finite cycle-free graph is proper, then 
every symbol configuration a e C maps to a unique state 
configuration s G 5, so the realization is one-to-one, and thus 
observable. It then follows from the dualities of Theorems 
1 and 4 that under the same conditions a trim realization is 
controllable. D 

Thus a minimal (trim and proper) finite cycle-free linear or 
group realization is observable and controllable. However, the 
converse is not true: 

Example 1 (cont.). The trellis realization of Fig. HJa) is 
observable, but improper The dual realization of Fig. 2Ic) is 
controllable, but not trim. D 

Thus in this context properness is stronger than observ- 
ability, and trimness than controllability. Properness implies 
invertibility not only globally, but also for fragments of a 
realization created by cuts. Similarly, trimness implies state 
reachability not only globally, but also for fragments. 

C. Controllability test 

For a direct test of controllability, we regard J|^ C:^ as a 
subspace of ^ x 5 x 5, as in Fig. [Ub). We observe that the 
subspace /C = {(0,s, — s) g Hi'^i^} i^ isomorphic to the 
unobservable state configuration subspace 5" = {s e 5 | 
(0, s, — s) G Hi ^t} of '^he dual realization. Then we obtain: 

Theorem 6 (Controllability test). The size of the behavior 
*8 of a linear or group realization with state configura- 
tion space S = 111 '^i ^^^ constraint codes Ci is |!B| — 
(nj|C»|)|5"|/|5|; or, in the linear case, dim 03 = Y.^dm).C,+ 
dim iS" — dim S. Thus the realization is controllable if and only 
if |»| = (n, \C.\)/\S\ = (n. |C.|)/(n, l'5,l); oreqmvalently, 
in the linear case, if and only if 



dim*B = y dimC; 



dim S —y dim Ci 



^dim^j. 



Proof. As in Section III-II define *8^ as the image of the 
homomorphism S : J|^ C^- -^ A x S such that (a, s, s') i— > 
(a, s+s'); then 03^ is the orthogonal code to *B. Since K. is the 
kernel of S, we have *B^ ^ (niCj^)//C by the fundamental 
theorem of homomorphisms, so |*B^| = (Hi l^i^D/l^l- Since 
1*8! = \A\\S\/\^\ Id = |^W||5W|/|C,^| and \1C\ = |5"|, 
and moreover, as a consequence of the normal degree re- 
strictions, nj"4(')| = 1^1 and n,;l'5''^| = \S\^, we have 
1^1 = {Y{i\Ci\)\S'^\/\S\\ or, in the linear case, dim«8 = 
XiidimCi + dim5" - dimS. Thus |<B| > {]\,\C,\)/\S\, 
with equality if and only if |5"| = 1; i.e., if and only if 
the realization is controllable. D 

Example 2 (cont.). In Fig. IMb), we have dim*B° = 1, 
X^i dimCj^ = 3 and ^- dim5i = 3; therefore by Theorem 6 
this realization is uncontrollable. However, for Fig. |6}a), we 
have dim 25 = 3, ^^ dimC; — 6 and ^^ dimiSi = 3, so this 
realization is controllable. D 

If a linear realization is both observable and controllable, 
then we have dimC ~ ^^dimCi — X]idim5j, since 



dimS — dimC. As Kashyap lfT4ll observed, this implies that 
for minimal finite cycle-free linear realizations, the two com- 
plexity measures ^^dimCi and X^idimiSj are equivalent. 

D. Generator and parity-check realizations 

We now investigate the observability and controllability of 
two standard types of realizations. 

A generator realization of a linear code C C F" is 
specified by a set of I generator n-tuples gi e F", such 
that C is the set of all linear combinations a = ^^ cti^i 
as the coefficient ^-tuple a runs through F^. The generators 
are linearly independent if dimC = i. A corresponding 
generator realization has (up to) n internal replicas aik of 
each of the I free coefficients a^, constrained to be equal by 
£ equality constraints an = • • ■ = aj„; n external variables 
Ak, and n linear constraint codes that enforce the constraints 
a/c = Y.i ctikQik = Yjt cngik- (If gik = O, then the replica atk 
may be omitted.) 

A parity-check realization of a linear code C C F" is 
specified by a set of r check n-tuples h^ G F", such that 
C is set of all n-tuples a G F" that are orthogonal to all check 
n-tuples hj. In other words, C is the orthogonal code to the 
linear code C^ generated by the r check n-tuples. The checks 
are linearly independent if dimC^ = r, or equivalently if 
dim C = n — r. In the corresponding parity-check realization, 
there are n external variables Ak, n one-dimensional constraint 
codes that generate (up to) r multiples akhjk of each of the 
external variables ak, and r zero-sum constraint codes that 
enforce the parity checks = J^k'^khjk. (If hjk — 0, then 
the multiple akhjk may be omitted.) 

As expected, generator and parity-check realizations are 
duals: 

Theorem 7 (Generator/parity-check reaUzation duaUty). 

Let a generator realization of a linear code C be specified by 
£ generator n-tuples g^; then its dual realization is the parity- 
check realization specified by the same set of £ n-tuples as 
checks, which realizes the orthogonal code C^. 

Proof. To prove that the realizations are duals, one need only 
check that: 

(a) the orthogonal code to the code generated by the equality 
constraint an = ■ ■ ■ = ain is the code generated by the 
zero-sum constraint ^j, onk — 0; 

(b) the constraint code defined by a check ak — X]j OiikQik 
on the variables ak, aifc, . . . , aik is the dual code to the 
code {(flfe, -akgik, ■ • • , -a-kgek) ■ ak G A}- □ 

Example 3 (binary Reed-Mull er code). The five binary 8- 
tuples 11110000,00111100,00001111, 11000011,01011010 
form a set of five linearly dependent generators for a binary 
linear block code C of length 8 and dimension 4, namely the 
(8,4,4) first-order Reed-Muller (RM) code. The correspond- 
ing generator realization is shown in Fig. |2la). 

Since C is self-dual (i.e., C^ = C), the same five binary 8- 
tuples form a set of linearly dependent checks for C. Fig. [TJb) 
shows the parity-check realization based on these five check 
8-tuples, which is the dual to that of Fig. [Tja). D 
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Fig. 7. (a) generator realization of (8, 4, 4) RM code; (b) parity-check 
realization of same code. 



Example 2 (cont.). We note that the primal realization of Fig. 
|6fa) is a generator realization based on the three linearly de- 
pendent generators 110, Oil, 101, whereas the dual realization 
of Fig. |6jb) is a parity-check realization based on the same 
three 3-tuples as checks. D 

We now settle the observability and controllability proper- 
ties of such realizations as follows: 

Theorem 8 (Observability/controllability of generator and 
parity-check reaUzations). A parity-check realization is ob- 
servable, and a generator realization is controllable. A gen- 
erator realization is observable if and only if its generators 
are linearly independent, and a parity-check realization is 
controllable if and only if its checks are linearly independent. 

Proof: In a parity-check realization, the internal variables are 
all multiples of external variables; therefore if all external 
variables are zero, then all internal variables must be zero. 
Thus a parity-check realization is necessarily observable. 
By observability /controllability duality, a generator realization 
must therefore be controllable. 

In a generator realization, the internal variables are replicas 
of the £ free coefficients, so s 7^ if the coefficients are 
nonzero. Thus there exists a nonzero configuration (0, s) — 
i.e., the realization is unobservable — if and only if there 
exists some nontrivial linear combination of the generators 
that equals the zero codeword G C, which is true if 
and only if the generators are linearly dependent. By ob- 
servability/controllability duality, a parity-check realization is 
controllable iff its checks are linearly independent. D 

It is also interesting to determine controllability using the 
test of Theorem 6. In a generator realization, there are £ 
equality constraints of dimension 1, and n constraint codes 
of total dimension e, where e = dim S is the number of one- 
dimensional internal variables. Thus ^- dimCi — £ + e. Since 
the internal variables are replicas of the £ free coefficients, 
the dimension of the behavior *B is £. Thus dim 58 = 
J2i dimCj — dimiS, so a generator realization is controllable. 
In a parity-check realization, there are n equality constraints 
of dimension 1 , and r single-parity-check constraints of total 
dimension e — r, where again e = dimiS is the number of 
internal variables. We thus have J^i dimCi — n + e — r. By 



Theorem 8, the realization is observable, so dim*B = dimC. 
By Theorem 6, the realization is thus controllable if and only 
if dim C = n ^ r. 

The fact that a parity-check reahzation is controllable iff its 
checks are linearly independent nicely illustrates our definition 
of "controllable" as "having independent constraints." 

E. Unobservable or uncontrollable => locally reducible 

We now show how, given an unobservable linear realization 
of a linear code C on a finite graph Q, we may reduce the 
dimension of the behavior 05 by trimming one state space, 
without changing the realized code C. Thus an unobservable 
realization is locally reducible. It follows that the dual uncon- 
trollable realization may be locally reduced by merging the 
corresponding dual state space. A similar result was found by 
Koetter |16, Lemma 8]. 

For brevity and clarity, we assume that the unobservable 
realization is linear; the group case is similar We select any 
state space Sj such that Sj ^ in some nonzero configuration 
(0,s) G 05. We choose a basis {gji} for Sj with gji — Sj. 
The coordinates of Sj in this basis are thus 10 ... 0. 

Define the subspace Tj C Sj as the set of all states in Sj 
whose first coordinate in the chosen basis is 0, and trim the 
realization by restricting *B to the subbehavior 05' consisting 
of those configurations that pass through a state in Tj ■ We then 
replace Sj by Tj, reducing the state space dimension by one. 

The original unobservable configuration (0, s) is then not in 
the trimmed behavior *8', since the first coordinate of Sj is 1. 
However, given any (a, s') £ 05, the entire coset {(a, s'+as) : 
a G F} is in *B and thus realizes a. Since Sji = 1, there is 
precisely one element in this coset whose state at time j has 
first coordinate zero. Thus the trimmed realization still reahzes 
every a G C. 

In the dual realization, the corresponding local reduction is 
the merging of the states in Sj to their cosets in Sj/{Tj)^ via 
the natural map. (In the coordinate representation, this amounts 
to deleting the first coordinate of Sj.) Since the trimmed primal 
realization still generates C, the merged dual realization must 
still generate C^. Thus we have proved: 

Theorem 9 (Local reducibility of unobservable or uncon- 
trollable realizations). An unobservable Unear realization on 

a finite graph Q with an unobservable configuration (0, s) G S 
may be locally reduced by trimming any single state space in 
the support of s. The dual uncontrollable realization may be 
locally reduced by the dual merging operation. D 

Example 4 (tail-biting trelUs realization; from llTTI . IIT2I 
via (lOi). The linear tail-biting trellis realization shown in 
Fig. [HJa) realizes the binary linear (5,3) block code C = 
(01110,00011,10001). Since it has a nonzero configuration 
(0, s), this realization is unobservable. It has five state spaces, 
which have been coordinatized so that the state values along 
the nonzero trajectory (0, s) are either 10 or 1. It has five con- 
straint codes Co = (01|l|0,10|0|l),Ci = (0|1|1,1|1|0),C2 = 
(0|1|01,1|0|10),C3 = (00|1|01,01|1|10,10|0|10), and C4 = 
(00|1|01,01|1|00, 10|0|10). This realization is the prod- 
uct of one-dimensional realizations of the four generators 
01110,00011, 10001,01 no, with the indicated circular 
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spans, and correspondingly its behavior S has the following 
four generators: 



5*0 Aq Si A\ S2 



A. 



Aa 



00 

00 

01 1 
10 1 



11 

00 
00 
01 



1 00 

1 01 1 

00 1 

1 10 




iL, - - . 1 . 

^ . o\ 1 ./o /Viu 



1.1. 

. n\ n ^ . n . n . 
(a) (b) 

Fig. 8. (a) unobservable tail-biting trellis realization; (b) uncontrollable dual 
realization. 

The dual realization to that of Fig. [8ja) uses the same 
variable alphabets and state spaces, and the orthogonal con- 
straint codes C^ = (01|l|0,10|0|l},CiL = (1|1|1),C^ = 
(0|1|01,1|0|10), C^ = (11|0|10,10|1|11), and C^ = 
(10|0|10, 01|1|01), as shown in Fig.lSb). This dual realization 
realizes the orthogonal (5,2) code C^ = (10111,01100). It 
is a product of realizations of generators 10 111 , 01100 with 
the indicated circular spans, the second span being the entire 
time axis. Its behavior !B° is correspondingly generated by the 
following two generators: 

5*0 Aq Si Ai S2 A2 S3 A3 S4 A4 



01 


1 











1 


01 


1 


01 


1 


10 





1 


1 


1 


1 


11 





10 






By the test of Theorem 6, this realization is uncontrollable, 
since dim«B° = 2,X;.,dimC,^ = 9 and dim 5 = 8. (Notice 
the two disjoint subbehaviors.) 

To reduce the unobservable realization of Fig. HJa), let us 
trim the state space So- The resulting constraint codes are 
then Co = (1|1|0),C4 = (01|1|0, 00|1|1); note that both are 
not trim. 




Fig. 9. State-trimmed realization, and state-merged dual realization. 



The dual realization merges states in Sq 



constraint codes Cn 



(1|1|0,0|0|1), C4^ 



yielding the dual 

= (01|1|1,10|0|0) 



(both improper). The resulting merged realization is shown in 
Fig. |9jb); it is controllable but improper. 

If we trim the realization of Fig. |9ja), then we obtain 
a minimal (hence irreducible) realization with state space 
dimension profile (1, 0, 1, 1, 1), shown in Fig. [TOl a). This is a 
product realization with generators O HI O, 00011, l_000i, with 
the indicated circular spans. 



1.0.0 



1.1. 



, oyf) . ./oyrK. . oyo .^n\ o . o . 
(a) (b) 

Fig. 10. Trimmed state-trimmed realization and its dual, both minimal. 



The dual minimal (hence irreducible) realization is shown 
in Fig. [rolb). This is a product realization with generators 
01100,10111. D 



F. Unobservable and uncontrollable tail-biting trellises 

We now show that the disconnected subtrellises that can be 
observed in Figs. |6jb) and [8lb) are characteristic of uncon- 
trollable linear tail-biting trellis realizations. This property is 
reminiscent of the similar classical uncontrollability property 
for conventional linear state (trellis) realizations. 

Theorem 10 (Uncontrollable tail-biting trellis realizations). 

If a trim linear tail-biting trellis realization is uncontrollable, 
then its behavior consists of disconnected subbehaviors. 

Proof. We first observe that a proper linear tail-biting trellis 
realization is unobservable if and only if it has a nonzero 
trajectory (0,s) for which Si ^ for all i, since propemess 
implies that no constraint code can have a word of the 
form (0,0, Si+i) or (si,0,0). By trim/proper and observ- 
ability/controllability duality, a trim linear tail-biting trellis 
realization is therefore uncontrollable if and only if its dual 
realization has such a nonzero trajectory (0, s). 

As in Section HV-EI we choose a basis {gu} for each dual 
state space Si with gu — Si. The coordinates of each Si in 
this basis are then 10 ... 0. Moreover, if we choose a dual 
basis for each primal state space Si, then the inner product 
of elements of Si and Si is given by the dot product of their 
coordinate vectors over F. Thus the set s^ C Si of primal 
states orthogonal to Sj is the subspace % of Si consisting 
of elements whose first coordinate is zero. Moreover, the |F| 
cosets of Ti in Si are the subsets of Si whose first coordinates 
are equal to a certain value of F. 

Now each dual constraint code (Ci)^ contains an element 
(si,0,Si+i) with Si and Si+i having coordinates 10... 0. It 
follows that if (s^, ai, Si+i) is any element of Ci, then (using 
the convention that the term (s^+i , s^+i ) is negated in the inner 
product) 

= {{si,ai,Si+i),{si,ai,Si+i)) 
= (si,Si) + (ai,ai) - (si+i,s,j+i) 

= Sii — Si+1,1. 

It follows that in any trajectory (a, s) G 25, the first 
coordinates of the state variables must be equal: sn — s^+i 1. 
Moreover, since this holds for all i, all first state coordinates 
are equal in all trajectories (a, s) G *B. Thus the state spaces 
Si are partitioned by their first coordinates into |F| cosets of 
Ti ~ sj-, such that state transitions are possible only between 
cosets with the same first state coordinates. It follows that the 
|F| subbehaviors of *B comprising trajectories that have the 
|F| different possible values of the first state coordinate are 
disjoint. D 
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Using the coset partitioning of this proof, we define the zero 
subbehavior !Bo as the zero coset of S, which contains the 
zero trajectory (0, 0). Each of the |F| cosets of *Bo in *B then 
corresponds to the trajectories in *B that go through states with 
a particular value of the first coordinate. 

Example 2 (cont.). In the uncontrollable linear tail-biting trel- 
lis realization of Fig. |6jb), the zero subbehavior comprises the 
all-zero trajectory; the all-one trajectory is its coset. D 

Example 4 (cont.). In Fig. [SJb), the zero subbehavior com- 
prises the two lower trajectories; the two upper trajectories 
comprise its coset. D 

To better understand this result, and to sketch a proof 
of its converse, we recall the fundamental structure theorem 
of Koetter and Vardy ifTTl . ifTSJI : every linear state-trim and 
branch-trim tail-biting trellis realization is a product realiza- 
tion. In other words, the behavior *B is the product of dim 58 
one-dimensional tail-biting trellis realizations. Proper realiza- 
tions are the product of proper one-dimensional realizations. 
Unobservable realizations are those with dim*B > dimC 
generators. 

One-dimensional reahzations may have state support of 
size I < n or size n (called "degenerate support"). By the 
controllability test of Theorem 6, the behavior is controllable 
in the former case (since ^jdimCi — ^ + 1, whereas 
X^idimiSi = I and dim*B =1), but uncontrollable in the 
latter (since ^-dimCi = ^^dimiS^ — n). It is easy to see 
that a product of one-dimensional realizations is uncontrollable 
and has disconnected subbehaviors if and only if a component 
has degenerate support. 

Example 4 (cont.). In Fig. [8|b), there is only one possible 
generator whose state support is not the entire state time axis, 
namely the generator of the zero subbehavior; the other gen- 
erator must be taken from the nonzero coset, and necessarily 
has degenerate support. D 

We remark that uncontrollable (but observable) tail-biting 
trellis realizations in general have too few "starts" and "stops" 
(transitions between a zero state and a nonzero state); e.g., Fig. 
[Htb) has only one "start" and "stop," even though dimC"*- = 2. 
Conversely, unobservable (but controllable) tail-biting trellis 
realizations have too many "starts" and "stops;" e.g., Fig. Oa) 
has four "starts" and "stops," even though dimC = 3. The 
difference is the dimension dim 5" of the dual unobservable 
subspace in the former case, and dim 5" = dim *B — dim C 
in the latterQ 

G. General unobservable and uncontrollable realizations 

In this subsection, we consider a general unobservable linear 
realization on a finite graph Q, and its dual uncontrollable 
realization. We will see that uncontrollability does not manifest 
itself in such an obvious way as in the tail-biting case. 

■* Koetter and Vardy make the following related observations in |18 
Theorem 4.6 ff.]. Define k = J^^ dimCi — J^^ dim5i. A linear tail-biting 
trellis realization that has no degenerate-support generating trajectories (i.e., 
that is controllable, so fc = dim "S by Theorem 6) satisfies k > dim C, with 
equality if the reahzation is observable. If the realization is uncontrollable 
but observable, then k < dimC (i.e., J^^ dimCi — Y2i dim5i < dimC = 
dim®). 



We consider a proper unobservable realization with nonzero 
configuration (0, s) G S. Then, by properness, the projection 
s^'^ of the state configuration s on the state alphabet S^^^ of 
any local constraint code Ci must have either or 2 or more 
nonzero components. 

We will define a generalized cycl^ as a subgraph Q' C Q, 
not necessarily connected, in which all vertices have degree 2 
or greater; i.e., there are no isolated (degree-0) or leaf (degree- 
1) vertices Thus the support of a nonzero configuration 
(0, s) G 25 must be a generalized cycle. 

Fig. [TT] shows a simple generalized cycle. In general, a 
generalized cycle is a union of cycles. 






Fig. 1 1 . Example of a generalized cycle. 

Again, we may choose a basis {gje} for each state space 
Sj in the generalized cycle with g^i = Sj, so that every state 
variable Sj has first coordinate 1 and remaining coordinates 
equal to 0. Then the one-dimensional subspace of *8 generated 
by the nonzero configuration (0, s) is the set of all (0, as) 
in which the first coordinates of the state vector s are all 
equal to a G F on the generalized cycle, and all remaining 
state coordinates are zero. Hence the first coordinates are 
realized by a simple repetition realization C over F defined on 
the generalized cycle Q in which all constraints are equality 
constraints, as illustrated in Fig. [T2ja). 

For a realization of the dual uncontrollable code, we choose 
a dual basis for each dual state space Sj, so that the inner 
product of elements of Sj and Sj is given by the dot product 
of their coordinate vectors over F. Since all configurations 
(a, s) G S° in the dual realization must be orthogonal to the 
configuration (0, s), it follows that the first coordinates Si in 
any such configuration must lie in the orthogonal code C-^ 
to the repetition code C. Thus C^ is realized by the dual to 
the repetition realization, in which every equality constraint 
is replaced by a zero-sum constraint, and a sign inverter 
(represented by a dot) is inserted in the middle of every edge, 
as in Fig. [TSl b). 

Such a dual realization may be further simplified as follows. 
Since the concatenation of a degree-2 zero-sum constraint and 
a sign inverter is an equality constraint, and since a linear 
constraint surrounded by sign inverters is equivalent to the 
constraint without the sign inverters, we can replace a system 
such as that of Fig. [T2l" b) by an equivalent system such as that 
of Fig. [TUc). 

^Interestingly, Chemyak and Chertkov introduce the same concept, which 
they call a "generalized loop," in their loop calculus |2|, |3|. They express 
the partition function of a finite graph as a finite series, whose leading term 
is a Bethe approximation, and whose higher-order terms correspond to the 
"generalized loops" of the graph. 

^Such a graph is sometimes called a "2-core;" however, we avoid this term 
because a 2-core is implicitly a maximal connected subgraph with no leaf 
vertices. 
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(a) (b) 

Fig. 12. Repetition realization defined on a generalized cycle, its dual, and equivalent dual. 



(C) 



In this example, the dual behavior consists of all configu- 
rations in which the values of the first state coordinates are 
constant across the top, middle and bottom chains of degree- 
2 vertices, say to at,am,ab, respectively, and furthermore 
at + ftm + flb = 0. Thus in general 0, 2 or 3 of the values 
at,a,n,ab are nonzero. The subgraph Q" C Q' defined by 
the nonzero labels is therefore either a set of isolated vertices 
(i.e., the graph comprising all vertices in Q', but no edges), 
or a generalized cycle, which can be either one of the three 
cycles embedded in Q', or the generalized cycle Q' itself. 

In the binary case (F = F2), a zero-sum codeword must 
have an even number of ones. Therefore the support Q" of a 
nonzero configuration in a dual to a repetition realization must 
be a generalized cycle in which every vertex has even degree, 
which is called an Eulerian graph. For example, in the above 
example, Q" must be either a set of isolated vertices, or one 
of the three cycles embedded in Q' . It is easy to see that in 
general an Eulerian graph is a union of edge-disjoint cycles. 

In summary: 

Theorem 11 (Supports of unobservable and uncontrollable 
behaviors). Given a proper unobservable linear realization on 
a finite graph Q with an unobservable configuration (0, s) G 
*8, the support of (0, s) must be a generalized cycle Q' '^ Q. A 
repetition realization as in Fig. [T2l a) determines the possible 
values of the first state coordinates for the subspace generated 
by (0, s). Its dual realization as in Fig.[T2jb) or (c) determines 
the possible values of the first dual state coordinates on Q' of 
all configurations in the behavior 03° of the dual uncontrollable 
realization. If F = F2, then the support of any such dual 
configuration is an Eulerian graph Q" C Q'. D 

This theorem yields an alternative proof of Theorem 5, since 
it implies that a proper finite cycle-free linear realization is 
observable, and thus by duality a trim finite cycle-free linear 
realization is controllable. 

If a dual constraint code C^- has degree two in the gener- 
alized cycle, then it follows from this development that the 
first coordinates of the two dual state variables involved in 
C^ must be equal. Thus, for any chain of degree-2 nodes in 
a generalized cycle, all first state coordinates must be equal 
throughout the chain. Thus when the generalized cycle is 
simply a single cycle, we retrieve the results of the previous 
subsection for tail-biting trellis realizations. 

Example 3 (cont.). Consider again the unobservable generator 
realization of the (8, 4, 4) RM code C shown in Fig. |3a), and 
its uncontrollable dual parity-check realization shown in Fig. 

Etb). 



A repetition realization on the graph Q' supporting the 
nonzero configuration (0, s) of Fig.|7ja) is shown in Fig. fTsl a). 
In this case Q' is itself an Eulerian subgraph, with all vertices 
having even degree. 





Fig. 13. (a) repetition realization on unobservable support graph Q'; (b) dual 
realization. 

The dual realization on Q' is shown in Fig. [T3l b). 
Since all state variables in Fig. |2lb) are binary, all 
state configurations must satisfy the constraints of Fig. 
[TSl b). There are 32 such state configurations, whose 
supports are the 32 possible Eulerian subgraphs of Q'. 
The 32 symbol 8-tuples corresponding to these 32 state 
configurations in Fig. |2lb) form a linear (8, 5) supercode 
(11000000, 00110000, 00001100, 00000011, 01010101) 
{00,11}'^ + {01, 10}^ of the (8,4,4) RM code C. The 
additional constraint imposed by the last equality constraint 
of Fig. |3b) together with these constraints specifies C. 

We observe that in the uncontrollable realization of Fig. 
\T3\ h) there is no partition of the behavior *B, the state spaces 
Sj, or the constraint codes Ci into disjoint, disconnected 
subsets as in the case of uncontrollable tail-biting trellis 
realizations (Theorem 10). In particular, all state variables 
are binary and are replicas of symbol variables, so for any 
pair of state variables that are replicas of different symbol 
variables, every possible pair of state values occurs in some 
valid configuration. Thus the effects of uncontrollability are 
not as easy to see as in the single-cycle case. D 

H. Decoding unobservable or uncontrollable realizations 

To decode a given linear code C, we may use any realization 
of C that we like, and any decoding algorithm based on that 
realization. In this subsection we will assume that the well- 
known sum-product (belief propagation) algorithm is used. 
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or any similar iterative message-passing algorithm. Would 
there ever be any advantage to using an unobservable or 
uncontrollable realization of CI 

It is hard to imagine that there would be any advantage 
to using an unobservable realization, because in any such 
realization every codeword a G C is realized by multiple 
configurations, and with the sum-product or any similar un- 
biased algorithm, the weights of all configurations realizing 
the codeword a will be the same. Thus an iterative decoding 
algorithm will never converge to a single configuration. 

On the other hand, suppose that we use an uncontrollable 
tail-biting trellis realization of C. As we have seen, such a re- 
aUzation partitions into multiple disconnected subrealizations, 
each realizing a distinct coset of a subcode of C. With the 
sum-product algorithm, each subrealization will be decoded 
independently of the others. It is possible that this could 
actually be helpful, because: 

• decoding of the "correct" subrealization (the one contain- 
ing the transmitted codeword) should be easier, because 
it realizes a subcode of lower rate than C {i.e., further 
from channel capacity); 

• decoding of the "incorrect" subrealizations hopefully 
should fail in a detectable manner, because all incorrect 
codewords are far from the correct codeword, so none 
should be close to the received sequence. 

So if we are able to detect and terminate nonconvergent 
decoding of the "incorrect" subrealizations, then we could 
come out ahead. 

However, similar advantages could be obtained by delib- 
erately choosing any realization of any linear subcode C of 
C, using "translates" of this realization to realize the cosets 
of C in C, independently decoding these translates to find 
the "best" codeword in each such coset, and then finally 
comparing these codewords to find the "best of the best." The 
only advantage that we can see of using an uncontrollable 
tail-biting realization rather than this more general approach 
is that in the former case a standard sum-product algorithm 
may be used with Uttle or no modification. 

Furthermore, for more general uncontrollable realizations 
such as the redundant parity-check realization of Example 
3, uncontrollability does not in general imply disconnected 
subrealizations. Thus we cannot straightforwardly generalize 
our above argument for uncontrollable tail-biting realizations. 

In practice, realizations of LDPC codes with redundant 
parity checks have been considered for various purposes, and 
shown to be advantageous. In the original difference-set cyclic 
codes of Tanner Il24l . redundant checks were included for 
implementation symmetry; subsequently, MacKay and Davey 
II2I I showed that redundant checks contributed to the good 
decoding performance of such codes. More recent theoretical 
and experimental work {e.g., [|26l . ifTSl . Il28l ) has shown that 
redundant checks can reduce the size of the fundamental 
polytope and therefore the number of pseudocodewords, thus 
improving performance. 

In summary, there is theoretical and experimental evidence 
that redundant parity checks can be helpful. However, we are 
unaware of any comprehensive study of this question. 



V. Conclusion 
For realizations on general finite graphs we have shown: 

proper i — )■ trim and observable i — > controllable. 

Furthermore, we have shown the following relationships be- 
tween the various classes of realizations. Each of the indicated 
containments is strict. 



Cycle-free graphs 



observable, controllable 

minimal 

= trim and proper 
= locally irreducible 



General finite graphs 



trim, proper, observable, controllable 



locally irreducible 



minimal 



We have shown that the behavior of uncontrollable tail- 
biting trellis realizations partitions into disconnected subbe- 
haviors, as with bi-infinite conventional trellis realizations. 
However, this phenomenon does not necessarily occur with 
realizations on more general graphs. This may be related to the 
fact that trellis realizations always have product factorizations, 
whereas general realizations do not. 

Finally, we have observed that whereas unobservability 
seems undesirable in practice, there is evidence that uncon- 
trollability need not hurt, and may even be advantageous. We 
recommend further research into this question. 

In a subsequent paper fTol, we will focus on tail-biting trel- 
lis realizations. We find further criteria for local reducibility, 
and ultimately give a necessary and sufficient condition for 
local reducibility of tail-biting trellis realizations, under a more 
refined definition of local reducibility. 
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Appendix A: Minimal cycle-free realizations 

MAY HAVE NO PRODUCT FACTORIZATION 

We now show by a simple counterexample that there 
is in general no "product factorization" for minimal cycle- 
free realizations, in contrast to trellis realizations. (A more 
complicated counterexample was given in [5,, Section VILA].) 

Example A (binary SPC code). Fig. [14] shows a simple 
minimal cycle-free realization of the (3, 2, 2) binary linear 
single-parity-check (SPC) code. There are three binary-valued 
symbols, and three binary-valued state spaces. Three con- 
straints are equality constraints of degree 2, and one constraint 
is a zero-sum constraint of degree 3. 



Ao 



Ai 



Si 



So 



S3 



A. 



Fig. 14. Cycle-free realization of (3, 2, 2) binary linear block code. 

The full behavior of this realization is generated by the two 
configurations below: 

Ai Si A2 S2 A3 S3 



1 


1 


1 


1 














1 


1 


1 


1 



Thus both generators are "active" for the state S2, even though 
its state space dimension is 1. 

By contrast, in a product factorization, the dimension of 
every state space is equal to the number of generators that 
are "active" at that time. For example, a product factorization 
would yield a behavior such as the one generated by the two 
following configurations: 

Ai Si A2 S2 A3 S3 



1 


1 


1 


10 














1 


01 


1 


1 



But such a realization is nonminimal, as we see either by 
comparison with the realization above, or since its top con- 
straint code includes the branch (02, S2) — (0, 11), and is thus 
improper 

By symmetry, it is clear that a similar situation occurs 
regardless of which two of the three nonzero configurations 
are selected as generators. D 



Appendix B: Summary of Koetter 111611 

We give here a brief summary of some of the main results 
of Koetter lfT6l . using the notation developed in this paper In 
particular, we represent normal realizations by normal graphs, 
rather than by "trellis formations." 

As in Section ULTI of this paper, Koetter (Lemma 1) charac- 
terizes the full behavior *B C y^ x 5 by its orthogonal code 
*8^ C AxS. (In our development, *8^ is defined as the image 
of the homomorphism E : Yli C:^ -^ A x S that is defined 
by the sum map (a, s, s') n> (a, s + §'); this is equivalent to 
Koetter's definition, because for any j at most one of Sj or s' 
can be involved in any (Ci)^.) The code C realized by *B is 
then the projection ^\a- 

Alternatively, Koetter (Theorem 2) shows that if the or- 
thogonal code («8°)^ C y^ X 5 to the dual behavior ®° 
is constructed in the same way from Yli Ci, then its cross- 
section ((^B")-*-):^ is equal to C. In our terms, a simple proof 
is that by projection/cross-section duality, the orthogonal code 
to ((*8°)-'-):^ is (*8°)|_^ = C^. Koetter gives a direct proof, 
and then uses this result to prove the normal realization duality 
theorem (Theorem 3). 

In his discussion of vertex merging, Koetter (Lemma 5) first 
shows that if a constraint code Ci is not proper at Sj, so T — 
{Ci):S- 7^ {0}, and Ci' is the other constraint code adjacent to 
Sj, then for any nonzero t <E Twe can add to Cf a word whose 
only nonzero element is Sj = t without changing the code C 
that is realized. The proof is that (*B°)^ is unchanged, hence 
the dual behavior *S° is unchanged, hence the dual realization 
still realizes C^, hence the primal realization still realizes C. 
This yields a merging procedure equivalent to that shown in 
our Fig. Iljb). 

Koetter (Lemma 6) then shows that (*8°)^ contains a 
configuration supported by a single state space Sj if and only 
if the dual realization is not state-trim at Sj, and that in this 
case Sj is mergeable. 

In Lemmas 7 and 8 and Theorem 9, Koetter shows that 
the other condition for mergeability of Sj is that the dual 
realization is not one-to-one (i.e., observable) and contains 
a nonzero configuration (0,s) e *B° such that Sj is not 
orthogonal to Sj. 

Finally, Koetter (Theorem 10) gives a polynomial-time 
algorithm for identifying mergeable state spaces. 
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